feat(webhook): recover lost auto-plans from check_suite.requested - #1002
Open
Kiran01bm wants to merge 4 commits into
Open
feat(webhook): recover lost auto-plans from check_suite.requested#1002Kiran01bm wants to merge 4 commits into
Kiran01bm wants to merge 4 commits into
Conversation
Kiran01bm
force-pushed
the
kiran01bm/wh-8a-inbox-retry-after-not-before
branch
from
August 15, 2026 10:53
b3629c8 to
3a4b3c1
Compare
Base automatically changed from
kiran01bm/wh-8a-inbox-retry-after-not-before
to
main
August 15, 2026 11:00
Kiran01bm
marked this pull request as ready for review
August 15, 2026 11:15
Kiran01bm
requested review from
JashLal,
eeSeeGee,
jayjanssen,
jemiahw and
morgo
as code owners
August 15, 2026 11:15
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
Every push to a PR branch produces two independent deliveries: pull_request (what auto-plan acts on) and check_suite.requested. When the pull_request delivery is lost upstream of the inbox, the PR sits blocked until the reconciler's next scan. Feed the check_suite signal into the durable inbox with a grace-delayed not-before time so the organic delivery wins the race, and synthesize a recovery delivery only for open PRs still at the suite head whose auto-plan coverage is missing. Recovery rows reuse the reconciler's deterministic GUID so the two producers dedupe naturally. Kill switch: WEBHOOK_CHECK_SUITE_RECOVERY=false.
GitHub names a same-repo head's open PRs in the check_suite payload, so a non-fork suite with an empty list has nothing to recover: drop it at ingress and never walk the open-PR listing for it — the walk now serves only fork heads. Replaces the default-branch filter, which wrongly dropped suites for PRs whose head is the default branch. Also documents the Check suite App event subscription, without which the feature receives no deliveries and is silently inert.
Contributor
There was a problem hiding this comment.
Pull request overview
Adds a durable, grace-delayed recovery path for GitHub check_suite.requested webhooks so SchemaBot can quickly converge “lost” pull_request auto-plan deliveries for open PR heads, instead of waiting for the next reconciler sweep. This builds on the durable inbox’s ability to defer dispatch via retry_after and adjusts lag/backlog metrics to measure from the time a row became dispatchable.
Changes:
- Route
check_suitewebhooks through the handler and durable dispatcher, enqueueingcheck_suite.requestedwith a not-before grace delay and synthesizing missing auto-plan deliveries when coverage is absent. - Extend the durable inbox/store to persist
retry_afteron create, make pending rows honor not-before times, and propagateClaimableSincefor correct dispatch-lag measurement. - Add metrics, docs, and tests covering deferred dispatch semantics and check-suite recovery behavior + kill switch.
Reviewed changes
Copilot reviewed 17 out of 17 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| pkg/webhook/README.md | Updates webhook event flow docs to include check_suite.requested recovery behavior. |
| pkg/webhook/handler.go | Adds check_suite event routing and handler configuration fields/options. |
| pkg/webhook/durable_dispatch.go | Measures inbox dispatch lag from ClaimableSince and dispatches durable check_suite events. |
| pkg/webhook/durable_dispatch_test.go | Updates test stores to mirror real insert/claim semantics (ID population, ClaimableSince). |
| pkg/webhook/durable_dispatch_metrics_test.go | Adds coverage that deferred rows measure lag from due time (not receipt). |
| pkg/webhook/check_suite.go | New durable check_suite.requested ingress + recovery synthesis implementation. |
| pkg/webhook/check_suite_test.go | New tests for ingress gating, grace deferral, PR resolution, synthesis, retries, allowlist + kill switch behavior. |
| pkg/storage/types.go | Documents/introduces WebhookEvent.ClaimableSince; clarifies backlog age semantics. |
| pkg/storage/storage.go | Documents new inbox semantics: pending not-before via RetryAfter and ClaimableSince on claim. |
| pkg/storage/internal/sqlstore/webhook_events.go | Persists retry_after on insert, gates pending claimability on not-before, derives ClaimableSince, updates backlog-age query. |
| pkg/storage/internal/sqlstore/webhook_events_test.go | Adds SQL-store tests for pending not-before behavior, backlog-age basis, reopen behavior. |
| pkg/storage/internal/sqlstore/sql_helpers.go | Adds nullTimePtr helper for consistent nullable time parameter binding. |
| pkg/serve/serve.go | Wires new check-suite recovery handler option via env kill switch WEBHOOK_CHECK_SUITE_RECOVERY. |
| pkg/serve/serve_check_suite_recovery_test.go | Pins kill-switch contract (default enabled; false/invalid disables). |
| pkg/metrics/README.md | Documents new check_suite event type and schemabot.webhook.check_suite_recovery_total counter. |
| pkg/metrics/metrics.go | Clarifies inbox depth/age/lag semantics with deferral; adds check-suite recovery outcome counter. |
| docs/github-app-setup.md | Updates GitHub App setup docs to subscribe to Check suite + Check run + Pull request for the new behavior. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Align the malformed-payload test with the driver's dead-letter contract: a deterministic decode failure is marked failed_permanent, not retried into the plain failed state.
Kiran01bm
force-pushed
the
kiran01bm/wh-8c-check-suite-recovery
branch
from
August 15, 2026 11:34
1903d88 to
e1db01e
Compare
…_suite deliveries The recovery and durable-dispatch kill switches shared one guard that always reported recovery as disabled, sending triage to the wrong switch when only dispatch was off.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Feed GitHub's
check_suite.requesteddelivery into the durable inbox as a grace-delayed, redundant convergence signal that recovers lost auto-plan deliveries for open PR heads. Stacked on #999 (needsCreatepersistingretry_after); only the last commit is new here.Why
Every push to a PR branch produces two independent deliveries:
pull_request(what auto-plan acts on) andcheck_suite.requested. When thepull_requestdelivery is lost upstream of the inbox, the check suite sits queued with zero check runs and the PR is blocked with no recourse until the reconciler's next scan. The check_suite delivery is GitHub's purpose-built "populate your check runs for this SHA now" signal — handling it converges a lost auto-plan in minutes instead of waiting for the reconcile interval.What
check_suite.requestedonly (durable-gated, allowlisted, default-branch pushes filtered, fork heads pass through) and enqueues it with a not-before time — the recovery grace — so the organicpull_requestdelivery wins the race.pull_requests[]with mandatory current-head equality; bounded open-PR scan fallback for empty/fork payloads) and synthesizes a recovery delivery only for open PRs still at that head whose auto-plan coverage is missing.WEBHOOK_CHECK_SUITE_RECOVERY=false(default on, fail-safe to disabled on malformed values); honored at ingress and re-validated fail-closed at processing so already-queued rows stop synthesizing too.schemabot.webhook.check_suite_recovery_total(covered / synthesized / resynthesized / already_queued / no_open_pr).